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ABSTRACT 

This study examined the prevalence amd progression 
toward generally accepted goals of comprehensive student assessment 
programs (CSAP) using data from the Higher Education Panel's 
stratified sample of 455 colleges and universities, of which 357 
responded for a usable response rate of nearly 80%, The data 
indicated that about one-third of the nation's 2,600 nonspecialized 
degree-granting institutions had CSAPs in 1990. Among findings were 
the following: most institutions view their administration as the 
major force behind assessment? most institutions are addressing a 
broad range of student learning and development areas; 40% of 
institutions have established an assessment office; nearly 70% of 
respondents reported faculty as "moderately" or "strongly" involved 
in assessment planning or implementation; assessment leaders come 
from diverse backgrounds; 62% of respondents reported no detrimental 
effects or said it w.-.s too soon to cite liabilities; problems in 
implementation were student motivation/participation, financial 
support, faculty motivation/participation, availability of valid 
assessment methods, and use of results for program enhancement. 
Conclusions offer a 9-step CSAP development model based on findings, 
suggest some unanswered questions, and predict that the next 2 to 5 
years will be crucial to the assessment movement. Detailed 
statistical tables and a copy of the student assessment survey are 
attached. (JB) 
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SUMMARY 



Sludeni assessment in higher education 
has gained more and more attention since the 
publication of several national reports in the 
early 19K()s that directed the nation's attention 
t'' how well students were being educated in 
the country's colleges and universities. In the 
last several years national conferences have 
been held on Ihc issue and a variety of 
publications have appeared. 

The American Council on Education's 
annual Campus Trends study has carried 
"assessment questions" since I9S6. These 
have beeii, for the most part, general 
questions conceming any kind of assessment 
activity and planning. The special focus of 
this study is on "comprehensive" student 
assessment programs, and how they are 
progressing tov\ard the goals most higher 
education assessment experts and mandates 
are advocating today. Institutions with such 
programs identified themselves by indicating 
that their assessment programs consisted of 
two or more major components including the 
assessment of basic college level skills, 
general education, major field of study, 
careeer preparation, and/or personal growth 
and development. 

This publication presents data from the 
Higher Eulucation Panel's sample indicating 
that about one-third of the nation's 2.600 



nonspecialized degree-granting institutions 
had comprehensive student assessment 
programs (CSAPs) in 1990, The report not 
only presents some national "vital statistics" 
for these programs, but examines the extent 
to which certain issues are perceived as 
problems, indicates levels of faculty 
participation in the programs, and identifies 
unmet needs. 

The findings do not always agree with 
frequently expressed ideas conceming student 
assessment. For example, less than 10 
percent of the respondents considered (1) 
influence of outside agencies, or (2) the 
misuse of program results by outside 
agencies or the media to be "severe" 
problems. Regarding faculty participation, 
nearly half of the respondents reported faculty 
to have been heavily involved in the planning 
and designing of the assessment program, but 
twenty-five percent of the respondents 
identified greater faculty involvement as a 
major "unmet need." 

The authors offer a nine-step development 
cycle that evolves into an assessment/re- 
port /feedback loop. Finally, while 
identifying issues and circumstances that both 
help and hinder assessment, they note that 
solutions to even the toughest questions will 
likely by found. 
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HIGHLIGHTS 



Thirty percent of the nation's 2- and 4- 
year colleges and universities reported 
operating comprehensive student 
assessment programs in academic year 
1989-90. An additional 60 percent 
reported that they planned to establish 
such programs in th^ near future. 

Internal administrative decision -- not a 
requirement exiemal to ihe colleges - was 
most frequently cited as the reason for 
establishing such comprehensive student 
assessment programs. 

Multiple methods of assessment, most 
often including commercially and/or 
locally developed tests, are reported to be 
in place at most institutions with 
comprehensive student assessment 
programs. 

Seventy percent of the colleges with 
comprehensive student assessment 
programs reported faculty as 
"moderately" or "heavily" involved in 
planning and designing the program; less 
than half the institutions reported such a 
degree of faculty involvement in the 
direction or evaluation of the programs. 

Leaders of campus assessment efforts 
come from a wide variety of disciplines, 
although degrees in education, social 
sciences, and humanities are the mosi 
common. Frequently such leaders 
received their primary training in 
assessment through conferences and 
workshops. 

Very good or good ratings were given lo 
the current assessment programs at almost 
half of the institutions surveyed. 



• The problems related to assessment 
programs that appeared to be most severe 
in the minds of survey responaents were: 

student motivation/participation 

financial support 

faculty motivation/participation 

availability of valid assessment 

methods 

use of results for program enhance- 
ment. 

• Problems related to "outside interference" 
or misuse of assessment results by 
outside agencies appear to be more 
apparent than real, at least thus far. 

• The most frequently cited helpful decision 
in establishing a comprehensive student 
assessment program was strong faculty 
involvement. 

• The most frequently cited "unmet needs" 
for implementing successful assessment 
programs included: 

more resources 

better analysis and utilization of pro- 
gram results 

better assessme : methods 
greater faculty involvement 
better program planning and goal 
setting. 

• A nine- step development cycle for 
comprehensive student assessment 
programs is identified that includes 
planning, implementation, and feedback 
phases, 

• For the immediate future, the expansion 
of comprehensive student assessment 
programs seems assured. 



INTRODLCTION 



For the nearly twenty years since 
pioneering colleges took their first concerted 
steps toward systematically evaluating the 
quality of their academic programs, the higher 
education assessment movement has been 
growing in breadth and impact to the status of a 
national and even international phenomenon. 
During that period, the assessment of 
institutional effectiveness Las increasingly 
influenced not only college students, faculty, 
administrations, and curricula, but has also 
affected - and been affected by - accreditation 
agency standards and state executive and 
legislative education policies as well. And all 
indications point to assessment's continued 
growth as a major factor shaping the fuiure of 
higher education. 

Tracing the development of the 
assessment movement is difficult since, except 
in the cases of a few institutions, detailed infor- 
mation on its early period is lacking. But 
patterns of progress since the mid-1980's have 
been better documented, primarily through 
surveys by the American Council on Education 
(ACE) and the Education Commission of the 
States (ECS). We know, for exam.ple, that close 
to forty of the fifty states now actively promote 
higher education assessment - mostly through 
legislative or regulatory mandates - as compared 
to only three or four in 1985 (?^mney, 1990); 
and that of all colleges report having 
assessment activities underway today, up from 
55^/( just two years ago (El Khawas. 1990 and 
1988). 

Results like these have been invaluable in 
outlining the general profile of the assessment 
movement, but what of the all important 
operational details? How many colleges and 
universities are actually implementing 
comprehensive assessment programs, as 
required by most mandates? What indicators of 
quality are they measuring? What assessment 
methods are they using? How are assessment 
programs organized? And what sorts of higher 
educators are leading campus efforts? What are 
the key pitfalls in assessment, and perhaps more 
to the heart of the matter, how are faculty and 
administrations dealing with such problems? In 
other words, specifically whom is doing 
specifically what in higher education 
assessment, and how is it going? 



Now. for the first time, these and many 
other "nuts and bolts" questions have been 
posed to a stratified sample of 455 colleges and 
universities representative of over 26(K) two- and 
four-year post-secondary institutions in the 
United States. With nearly an 80^/ usable 
response rate (3^7 schools: sec the Technic . I 
Notes in Appendix B). this ACE-Winthi ^: 
survey constitutes the most thorough studs of 
higher education assessment pract ices 

accomplished to date/ 

Prior to this report, anyone in need of a 
reasonably detailed overview of what was 
happening in the assessment niox cmenl faced a 
daunting quest. Although valuable information 
abounds in the literature, ii is too widolx 
dispersed and varied in qua! it \ to be casilx 
located, digested, and/or utilized. There is no 
higher education assessment journal. on!\ one 
national newsletter, and no "handbook" 
compendium of relevant models, methods and 
materials. (Helpful efforts in this direction have 
been made by the American Association lor 
Higher Education's Assessment Forum, the 
federal Fund for the Improvement of Post 
Secondary Education's Assessment l^rogram 
Book, and the Assessment Resouice Center at 
the University of Tennessee-Know ille. but 
much remains to be done.) 

A few' assessment information 
clearinghouses ha\e sprung up. but thc\ are 
overextended, understaffed, and generally 
unable to provide integrated pictures of 
assessment efforts across models, institutions 
and topics. More in-depth knowledge of 
particular assessment approaches can be 
obtained via reports or workshops from 
individual institutions with more advanced 
assessment programs, but finding comparative 
frames of reference is time consuming and 
difficult. 

As another option, one might attend one 
or more assessment conferences w here a broader 
mosaic of experiences from across the higher 
education spectrum are available. But while all 
attendees will find some of what they need to 

« 

Nofe: Additiofial suppor! lor Ihis siutl\ proxulcti 
by South Carolina Commission on HilMki |-iliicalu>n 
and American Associalion lor llitihcr l^lucaiion tiranK 
lo Winihrop College. 



know, and a few will find much of what they 
need to know, most will leave the conference 
with their appetites for assessment information 
more whetted than satisfied. 

This study is an attempt to help narrow 
this higher education assessment 'information 
gap/' As compared to other sources, this study 
was designed to provide more information on 
the inner workings of assessment programs, yet 
on a much broader scale. The survey included 
both forced-choice and open-ended items (see 
Survey Questionnaire in Appendix A) divided 
into three sections covering overall program 
status, program personnel and organization, and 
program evaluation, respectively. Institutions 



with ?.ssessment programs in place were also 
asked detailed questions regarding the methods 
and strategies they are using, and the perceived 
strengths and weaknesses of their efforts thus 
far. 

It is intended that the present summary, 
as well as more in-depth treatments to follow, 
will constitute a useful frame of reference against 
which institutional assessment efforts - models, 
methods, options or issues - can be compared. 
By so doing, we hope the present results will 
facilitate the development of more data-based, 
better quality higher education assessment 
programs in the future, which is our primary 
goal for this study. 



ASSESSMENT PROGRAM STATUS 

According to our results, 309c of 
American colleges and universities report they 
are operating comprehensive student assessment 
programs (hereafter called CSAPs) as a primary 
indicator of institutional effectiveness. This 
much lower figure than those from the ACE 
Campus Trends and ECS surveys mentioned 
earlier may be explained by two major 
differences between this study and those. First, 
this survey specifically requested information on 
comprehensive assessment programs; i.e., 
efforts to measure undergraduate students' 
progress toward one to five categories of higher 
education objectives, usually in the arts and 
sciences. Institutions which would have 
responded affirmatively to other surveys on the 
basis of, for example, fledgling efforts or very 
limited assessment activities or primarily 
graduate-level measures, would likely respond 
in the negative here, thus lowering the 
percentage. A second important distinction of 
these results is that they address institutional- 
level assessment efforts rather than those at the 
state or program level, both of which would be 
expected to produce higher percentage 
responses. The percentage of states with at least 
one institutional assessment effort or the 
percentage of institutions with at least one 
program assessment effort (e g., even a single 
survey, basic skills test, credentialing exam, 
etc.) would obviously be much higher than the 
percentage of only those institutions 
implementing more comprehensive assessment 
programs. This study, therefore, rather than 
conflicting with previous results, is more 



complementary and elaborative, providing a bet- 
ter estimate of the numbers of broad institutional 
assessment programs that most assessment 
experts and mandates have been urging. 

The Overall Picture 

There was remarkable consistency in 
general assessment activity across the 
institutions surveyed, with the type or category 
of institutions having little bearing on the 
likelihood of a campus CSAP. (See Technical 
Notes for institutional category distinctions.) As 
is indicated in Table 1, approximately equal 
percentages of baccalaureate institutions, 
comprehensive colleges, and universities 
reported having a comprehensive program to 
measure student progress toward the 
institution's educational and related student 
development goals. A slightly higher percentage 
(32%) of public colleges and universities than 
independents (28%) appear to have such 
programs in place. 

Most institutions without an assessment 
program show a clear trend toward developing 
one. Twenty-seven percent (27%) of 
institutions who reported having no CSAP at the 
time of this survey have established specific 
timelines for program initiation and another 57% 
stated that assessment planning was underway 
although no implementation date had been set. 
This means that only 16% of institutions without 
a current program (approximately 11% of all 
institutions) report *^o current plans for 
comprehensive assessment. 
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This pattern of development from 1984, 
when less than 10% of all institutions surveyed 
reported having CSAPs, to 1990, when 89% 
report operational programs or plans for CSAPs, 
shows dramatic progress in the national trend to 
assess institutional effectiveness. These results 
are also in accord with the earlier mentioned 
ACE Campus Trends and ECS reports on the 
higher education movement's current status. 

When and Why Were Assessment 
Programs Begun?* 

Of those campuses reporting CSAPs, 
64% were begun within the past six years. 
While this time frame coincides with the spread 
of state government and regional accreditation 
assessment mandates, and such external 
pressures are commonly cited as a primary force 
behind higher education assessment, the reasons 
for CSAP initiation cited by institutions in our 
sample are considerably broader and more 
varied. (See Table 2.) 

Among the top nine reasons given for 
establishing a CSAP, a decision by campus 
administration was cited most frequently (73%). 
Accreditation standards at 46% and state 
mandates (including legislative and executive 
policies) at 39% were cited by less than half of 
the sample. Other intra-institutional factors were 
also clearly prominent, with faculty initiatives 
(41%), following "national trends" (39%). and 
following other institutions' examples (20%) 
making up four of the top five reasons ciied for 
beginning assessment programs. In the widely 
discussed matter of external versus internal 
incentives for assessment, internal factors 
appear to be as important, if not more important, 
than external pressures. The typical pattern 
portrayed in this study is that of institutional 
leaders becoming cognizant of assessment 
efforts on other campuses across the country and 
rallying the faculty to initiate their own local 
program, whether they are under an assessment 
mandate or not. 

What Is Being Assessed. ..And How? 

The advice of experts in higher education 
assessment has strongly emphasized the need for 

♦Nolc: From this poini on. pcrconlage responses and 
operational delails refer to the MW( of institutions 
surveyed who report having CSAPs already in place as of 
1990. (A few percentage totals may not equal exactly 
due to rounding.) 
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program models and methods tailored to the 
unique needs and characteristics of each 
institution. Yet, common institutional program 
objectives would suggest that common 
assessment components might be included in the 
efforts of most colleges and universities. To 
explore this question we divided institutional 
objectives into five categories (see Table 3), and 
asked institutions in the sample to indicate which 
components were included in their CSAP. 

Additionally, one of the most important 
objectives of this study was to find out what 
measurement methods were being relied on for 
student assessment. Are "off the shelf" 
commercial tests and surveys dominating the 
CSAP scene? How willing are faculty to devote 
the time and effort needed to develop "local" 
instruments? Have more competency-based 
"performance measures" made significant 
inroads? To what extent are available student 
records being used? In other words, we were 
very interested in learning just how institutions 
are measuring what they're measuring. (See 
Table 4.) 

The student learning and development 
objectives most frequently cited as a component 
of campus assessment programs was basic 
college-readiness skills. This component - 
defined as measuring student progress in such 
entry-level areas as reading, writing, and 
mathematics - was listed by 94% of institutions 
with assessment programs. Among these 
institutions, commercial tests were the most 
frequently cited method for assessing basic 
college readiness, with 82% reporting their use. 
Locally developed tests were utilized by 
approximately half of respondents who evaluate 
college readiness in their program, followed in 
frequency by available archival records (27%), 
performance-based methods (16% ), and student 
self-reporting (10%). The frequencies with 
which all individual methods were reported 
suggest that multiple methods of assessing this 
area are being used by many institutions. 

The second most frequently reported 
assessment program component was career 
preparation/alumni follow-up, which was cited 
by 76% of schools with CSAPs. Self-report 
was cleariy the "method of choice" for assessing 
career development and following-up with 
alumni, with no other method bi ng used by 
over 20% of the sample. 

The assessment of general 
education/liberal studies was reported as a 



component of the assessment programs of 61% 
of responding institutions. Commercial tests 
were again the most frequently cited 
measurement method, with about four in ten 
institutions reporting their use. Performance- 
based methods were second in frequency of use 
at 24%. followed by locally developed tests 
(15% of institutions), student self-reporting 
( 14%) and available archival records ( 14%). 

Approximately two-thirds of colleges 
and universities staled that major fields were 
being assessed in their CSAPs. Locally- 
developed tests and performance-based methods 
were more likely to be included in assessing 
major specialty areas than in any other program 
component, with 39% and 38%. respectively, of 
all institutions reporting their use. Commercial 
tests, used by 27% of the sample, were the next 
most frequently reported method of assessing 
majors, followed by available archival records 
(17%). and student self-reporting (12%). 
Again, these data suggest that multiple methods 
are being widely used to assess major fields in 



institutions, but it is not known to what extent 
multiple measures are typical within individual 
major departments. 

The last assessment component reported 
by a majority of respondents (65%) was the 
assessment of students' personal growth and 
development. This area was defined as 
"measuring values, attitudes, social development 
and/or other nonacademic changes attributable to 
the college experience." Commercial 
instruments were the most frequently reported 
method of assessment in this area (reported by 
29% of institutions), followed closely by student 
self-reporting at 24%. 

Taken as a whole, it is clear that a wide 
range of institutional objectives measured by a 
variety of methods are typical of the CSAPs in 
this study. And while traditional strategies like 
commercial tests, surveys, and student records 
are prominent, more innovative options such as 
locally developed instruments, performance 
appraisal, and qualitative student reports are also 
playing important roles. 
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CSAP Structure 

The organizational structure of any 
program, including a CSAP, is an important 
determinant of its long term stability and 
success. Furthermore, the administrative 
division to which a CSAP is assigned makes a 
strong statement about how assessment's role in 
higher education is aciually perceived on that 
campus. 

At this point assessment is apparently 
being seen as a primarily educational endeavor, 
with a clear majority of surveyed institutions 
(59Vf) giving an ai;ademic officer (i.e., vice 
president or dean of academic affairs) executive 
responsibility for their CSAP. Thirteen percent 
have put CSAPs in student affairs, 7% fall under 
institutional research, and 5% have added 
assessment leadership to the chief executive 
officer's duties. The remaining 17% have 
assigned their CSAPs elsewhere ( 14%) or are 
undecided/made no response (3%). (See Table 
5). 

Many key decisions and actions which 
ultimately determine assessment's success in 
enhancing the quality of institutional programs 
are also made by the individual(s) on the front 
lines charged with the day-to-day operations of 



CSAPs. We now briefly review who these 
"'assessment coordinators" are - regardless of 
specific title - and how their efforts are being 
organized. 

As Table 6 shows, a majority of all 
institutions (627f ) have put administrators in 
charge of ongoing program direction, as well as 
executive oversight duties, for their assessment 
efforts. Overall, only 20% have faculty 
member(s) coordinating CSAPs. 10% put 
operational responsibilities in the hands of a 
faculty committee, and 6% say "others" 
coordinate their CSAP, presumably referring to 
staff in student affairs and/or institutional 
research. 

A large minority of institutions with 
CSAPs (40%) have created an on-campus 
assessment coordinating office. The majority of 
those institutions report that such offices 
perform the following services: Consultation 
with faculty (83%), CSAP coordination (81%), 
hands-on assessment implementation (79%), 
consultation with administrators (72%), liaison 
with students (65%), planning (65%), research 
and development (58% ), and technical assistance 
(53%). Budgeting (46%) and assessment 
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program evaluation (38%) were also frequently 
cited assessment office functions. 

In addition to such student assessment 
coordinating centers, 85% of institutions with 
CSAPs report other assessment support 
structures, including faculty councils/committees 
(69% ) and/or administrative councils/committees 
(40%). Only 8% of all institutions report using 
assessment consultants, and only 3% of 
institutions with CSAPs directly involve student 
councils/committees in their operational 
structure. 

Faculty Roles In Assessment 

Most experts strongly advocate high 
faculty involvement in CSAPs to prevent higher 
education assessment from becoming - in 
appearance or in fact - merely an administrative 
function or an exercise in extemal accountability. 
Thus, we made a point in this study to examine 
the roles being played by faculty members in 
CSAPs. (See Table 7.) Thus far, faculty appear 
most prominently in the planning and designing 
of CSAPs. with 45% of institutions reporting 
faculty as "heavily involved" and another 50% 
citing slight to moderate faculty involvement in 
that program phase. In terms of directing or 
coordinating CSAPs. most institutions see that 
as an administrative role, with only 19% 
reporting heavy faculty involvement and 68% 
slight to moderate. Faculty participation 
increases again in CSAP operation and 
implementation, with 34% heavily involved and 
another 53% at least somewhat involved; but 
faculty play less of a role in CSAP program 
evaluation with only 21% heavily involved, ar d 
9% of institutions reporting no faculty role at all. 
While it is clear that at least some faculty input is 
common in almost all phases of CSAP activities, 
it does appear that the higher the level of 
assessment decision-making (i.e.. direction and 
evaluation), the lower the level of faculty 
involvement. 

Finally, in another example of 
conventional higher education w sdom 
apparently not being taken to heart in most 
CSAPs. only 10'-^ of the institutions sampled 
reported being members of an assessment 
consortium. 

Assessment Leaders: A Profile 

Higher educators who have reviewed the 
assessment literature or attended conferences or 
institutional workshops have observed an 



extraordinary diversity in the backgrounds of 
leading experts in the higher education 
assessment movement, and this seems to be true 
at both the national and institutional level. Yet 
the present survey is the first known effort to 
systematically stL:dy the assessment-related 
backgrounds of a vital group of such leaders; 
i.e., those responsible for campus CSAPs. 

Since CSAPs are relatively new 
programs, with no traditionally dominant 
disciplinary qualifications associated with leader- 
ship positions, we asked respondents to identify 
the individual leader "most involved with your 
campus assessment program." We then posed 
questions regarding that leader's academic 
discipline, degree level, position title, 
provenance, and assessment-related training and 
experience. In fact, we asked for the same 
information on the two most involved CSAP 
leaders to better encompass situations where 
assessment responsibilities are split or layered 
among more than one individual. 

The expected diversity is, in fact, evident 
in reported CSAP leaders' backgrounds (see 
Table 8). with at least one percent of the sample 
utilizing at least one coordinator from each of six 
broad discipline categories. Overall, in.stitutional 
assessment leaders are most likely to come from 
three disciplinary backgrounds: education 
(297r). social and behavioral sciences (25%), 
and arts and humanities (21%). The secondary 
CSAP leader profile is very similar, with only 
one other area (physical 
sciences/engineering/math at 7% for Leader #1 
and 12% for Leader #2) accounting for 5% or 
more of the total, 

There is considerable agreement among 
respondents that CSAP leaders are doctoral-level 
personnel (59% for Leader #1, 57% for #2), 
with only 2-year institutions using significant 
numbers of master's degree people, and less 
than 1% reporting CSAPs headed by bachelor's- 
level individuals. There is even more unanimity 
that assessment leaders for CSAPs are chosen 
from within the institution (86% for both 
Leaders #1 and #2) as opposed to bringing 
someone in from another college or other outside 
source. 

Surprisingly, when we inquired 
regarding the specific training and experience of 
CSAP leaders, only 16% reported having taken 
a degree program with an emphasis on 
assessment, and 4% reported no prior training or 
experience with assessment whatsoever. By far 
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the most common training experience reported 
by CSAP leaders was attending workshops or 
seminars (73%). with a degree in education or 
higher education (47%), and experience in 
assessment at the disciplinary program level 



(40%) or institutional level (20%), also being 
often reported as primary higher education 
assessment qualifications. A large number of 
institutions (45%) also cited other kinds of 
relevant experiences as important. 



ASSESSMENT PROGRAM EVALUATION 

Evaluating the effectiveness and 
efficiency of CSAPs should be a high priority as 
the higher education assessment movement 
progresses toward maturity. But, with the 
exception of the "first and second waves" of 
pioneer institutions in the 70's and early 80's, 
most colleges and universities are at much too 
early a stage of development to draw valid 
conclusions regarding the outcomes of their 
CSAPs. Thus, since product evaluation was 
unlikely to produce very meaningful results, this 
study focused on information regarding the 
process of assessment; what did respondents 
think was going right, what was going wrong, 
and what were they doing about both? 

Current Assessment Challenges and 
Steps Toward Resolution 

Institutions in the survey were provided 
a list of eleven potential challenges or problems 
related to implementation of a successful student 
assessment program and were asked to rate each 
on a five point scale from "no problem" to "very 
severe problem." They then were asked to 
identify the two most severe challenges. 
Respondents were also asked to list the two 
"most helpful" decisions they made in 
establishing their assessment program. Deci- 
sions identified as being "most helpful" were 
categorized to facilitate analysis, interpretation, 
and reporting. Problem rating totals are shown 
in Table 9, 

Considering all degrees of challenges 
together for the moment, a majority of all 
institutions identified the following as at least 
some problem: the availability of valid 
assessment methods and student 
motivation/participation (both 79%) tied for the 
most frequently cited problem, with coordination 
of all aspects of the CSAP (77%) and financial 
support for the CSAP (75%) closely following. 
Other widely reported problem areas were 
faculty motivation/participation (70%), using 
CSAP results for institutional program 
enhancement (69%). and analyzing CSAP 



results (67%). No other problem was cited by 
over half of the sample. 

When severity of problems is taken into 
account a slightly reordered pattern emerges. 
The number of problem areas rated as severe or 
very severe drops off dramatically and is headed 
by inadequate financial support (20%), 
availability of valid assessment methods (19%), 
student motivation/participation (14%), CSAP 
coordination ( 1 3%), and both faculty motivation/ 
participation and using CSAP results for 
program improvement (at 11%). A similar 
hierarchy was formed from responses to an 
open-ended question asking which two 
challenges were most severe. (See Table 10.) 
Significantly, over one-third of the sample did 
not identify any of the problem areas as "most 
severe." 

It is also of note that four widely 
assumed higher education assessment challenges 
were not seen as significant problems by our 
sample. In fact, at least half of the respondents 
reported them as "no problem" at all. These 
include administrative support (68%). misuse of 
results by the media or public (58%) or by 
regulating or accrediting agencies (55%), and 
undue influence on CSAPs by ouis'de agencies 
(50%). 

Thus, the survey found little, if any, 
support for the concern of some in higher 
education that assessment programs and their 
results are a vehicle for undue influence from 
outside agencies or an invitation for misuse of 
results by media, government, or other outside 
parties. Two percent (2% ) or less of responding 
institutions cited any of these problems as 
"severe," although many colleges and 
universities may be at such early stages of the 
assessment process that potential problems 
regarding results and their interpretation and 
dissemination have not yet manifested 
themselves. 

In terms of how CSAP leaders are trying 
to resolve these and other assessment 
challenges, we asked an open-ended question 



6 

hi 

ERIC 



regarding what decisions or actions taken by 
respondents they considered most helpful in 
setting up their programs. While a disappointing 
- or perhaps revealing - number of respondents 
(29%) gave no response, the following "best 
decisions'* were cited by a significant percentage: 
getting faculty involved (23%), integrating 
CSAPs with institutional planning (19%), 
careful analysis and feedback of results (15%), a 
determined commitment to and prompt 
implementation of the CSAP (14%), increased 
allocation of resources (12%), and local 
development of assessment methods (11%), 
Somewhat surprisingly, only 5% listed making 
local as opposed to externally mandated 
decisions concerning the program as one of their 
best moves. 

Nearly one in four institutions cited a 
decision 'most helpful" to assessment 
implementation that could not be classified into 
one of the major categories cited above. These 
decisions included such ideas as having ample 
time to plan and implement assessment, seeking 
student support, maintaining flexibility in each 
discipline's assessment program, obtaining 
assistance from consultants, and staffing an 
office specifically for assessment. The variety 
of decisions/suggestions received in response to 
this survey suggests a substantial degree of 
creative problem-solving at work in higher 
education assessment, as well as a need for 
approaches that are geared to the unique 
problems and characteristics of individual 
institutions. 

Perceived Benefits and Liabilities of 
Assessment 

With due caution regarding the 
previously noted fact that most CSAPs are in 
such a relatively early stage of development that 
respondents may be in a poor position to make 
retrospective judgments regarding their 
programs* ultimate worth, we next solicited 
open-ended comments on what the main 
beneficial and detrimental effects of assessment 
had been to date. Responses were then 
classified into the categories listed in Tables 1 1 
and 12, which contain the percentages of 
institutions citing particular positive or negative 
effects. 

Again, a significant number (26%) 
indicated that it was too soon to respond, but 
30% felt academic program planning had been 
enhanced, 29% saw assessment feedback 



improving students' efforts, and 3-107r said 
their CSAPs had increased their administration's 
cooperation and involvement in programs, 
upgraded standards, and improved faculty 
cooperation and morale. 

Regarding possible detrimental effects of 
CSAPs (Table 12) no single problem was cited 
by more than 9% of the respondents, a truly 
encouraging finding. Between 4-9% of 
respondents said assessment had produced or 
exacerbated the following problems: extra work 
for faculty and administration, increased drain 
on resources, "turf problems, and declines in 
morale. Of particular note is the fact that 53% of 
the sample institutions listed nothing in the 
"main detrimental effects" blanks, and 9% wrote 
in "no detrimental effects." With 62% of 
surveyed institutions citing no detrimental effects 
and only 26% reporting no present indications of 
beneficial effects, it seems safe to say that our 
sample sees CSAPs as "more worth than they're 
trouble," 

Current Unmet Needs and Assessment 
Program Evaluation 

The last open-ended survey item asked 
respondents to list their three greatest unmet 
needs in implementing successful CSAPs and 
utilizing the results for educational program 
enhancement. Again aggregating first, second, 
and third responses for all institutions, the most 
frequently cited needs were: more resources 
(42%), better analysis and utilization of CSAP 
results (33%), better assessment methods 
(28%), greater faculty involvement (28%), better 
CSAP planning and goal setting (28%), and 
"other" types of needs (15%). Relatively few 
respondents said that greater administrative 
interest (1 1%) or greater student interest (8%) 
were significant unmet needs, and 19% did not 
respond to this item. 

Finally - and again noting the interim 
nature of such judgments - we asked how 
respondents would rate the quality of their 
current CSAP. Only 9% said excellent, 49% 
said good or very good, 27% said only fair to 
poor, and 16% had no basis to judge or gave no 
response. Considering the scope of the 
challenges posed by higher education 
assessment under the best of circumstances - 
much less the difficult economic times many 
institutions have faced in the past few years - for 
over half of our respondents to rate their CSAP 
programs as good to excellent is taken as a very 



optimistic indicator of assessment's future 
development as a positive force in the 



improvement of higher education. 



SUMMARY AND CONCLUSIONS 

Summary 

This first in-depth study of the higher 
education assessment movement's status, on a 
representative stratified sample of campuses 
across the country, portrays a vibrant, growing 
phenomenon already having significant effects 
on academic programs. While other national 
studies have reported as many as H2% of 
colleges and universities having some 
assessment activities underway, our criteria for a 
comprehensive student assessment program 
(CSAP) - which are much closer, not 
incidentally, to the assessment recommendations 
of most experts as well as state and accreditation 
mandates - produced a much lower prevalence of 
30%. Still, we found that the number of schools 
either currently operating a CSAP or planning to 
do so in the near future stands at an impressive 
H9'7f. 

The impetus for assessment is seen by 
insdtutional representatives as being as much 
internally as externally based, if not more so. 
Despite the existence of higher education assess- 
ment mandates in close to forty states (and in 
6() /^ of the public institutions in our sample), a 
considerably higher percentage of institutions 
view their college or university administration as 
the major force behind assessment. Internal 
administrative decisions followed by 
accreditation standards, state mandates, and 
faculty initiatives, were the most frequently 
reported reasons for having begun a CSAP. 

The large majority of institutions with 
comprehensive assessment programs are 
addressing a broad range of student leaming and 
development areas including entry-level basic 
skills, general education, major fields, alumni 
follow-up/career development, and personal 
growth and development. Multiple methods of 
assessment, most often including commercial 
and locally developed instruments, also are 
reported to be in place at most schools. A multi- 
method approach is apparently more likely to be 
utilized for assessing basic college readiness 
(i.e., reading, writing, quantitative skills) and 
major fields of study than other assessment 
components. More innovative methods are also 
utilized by a significant minority of CSAPs. 



The importance of assessment is attested 
to by the fact that 40% of colleges and 
universities have established a coordinating 
center (i.e.. an "assessment office.") specifically 
designed to address this area. Most of these 
centers, as well as assessment programs in 
general, are run on a day-to-day basis by an 
administrator who usually reports to the chief 
academic officer or vice-president for academic 
affairs. Faculty groups such as academic 
councils are the most likely assessment support 
unit related to the CSAP on most campuses. 
Nearly 70% of our respondents reported faculty 
as "moderately" or "strongly" involved in 
assessment planning and implementation, with 
considerably less faculty involvement reported 
for important program supervision and 
evaluation functions. Most also reported no 
student involvement in CSAP development 
whatsoever. Thus, there is an opportunity and. 
if the advice of many assessment experts is 
followed, a substantial need for more faculty and 
student participation in CSAP operations on 
most campuses. 

The perception of many observers and 
participants of the higher education assessment 
movement is that assessment leaders tend to 
come from remarkably diverse backgrounds. 
This hypothesis was supported by the data from 
the present study Leaders of campus 
assessment efforts come from a wide variety of 
disciplines, although degrees in education, social 
sciences, and humanities are most common. 
Another perception, that a little experience and a 
few workshops or conferences can lead to one 
becoming viewed as an assessment "expert." 
was also borne out by our data. Although most 
assessment leaders possess terminal degrees, 
their credentials seldom include either formal 
education related to assessment or measurement, 
or significant prior experience in assessment. 
More than likely, such leaders have received 
their primary or sole training through 
conferences and workshops. This fact serves to 
emphasize the critical role of events such as the 
American Association for Higher Education's 
annual conference on assessment in higher 
education, state and regional meetings, and on- 
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campus workshops, as important means of 
sharing much needed expertise. 

At this point in time, institutional 
representatives are considerably more likely to 
cite beneficial effects than detriments of 
assessment, with 62% reporting no detrimental 
effects at all or saying it was too soon to cite any 
liabilities. The enhancement of academic 
planning and of student efforts and feedback 
appear to be the most cleariy recognized benefits 
of comprehensive assessment programs. The 
most frequently mentioned detriment, although 
cited by less than 10% of the sample, is the extra 
work for faculty and administrators that 
assessment programs tend to demand. 

The implementation of a successful 
CSAP presents many challenges for institutions 
and for administrators and faculty charged with 
being "assessment leaders/* Those problems 
which appear to be most severe in the minds of 
the institutional representatives completing our 
survey are student motivation/participation, 
financial support, faculty motiva- 
tion/participation, availability of valid 
assessment methods, and use of results for 
program enhancement. Yet, these problems do 
not appear to be considered major impediments 
to the assessment efforts of most institutions. 
Additionally, problems related to 'outside 
interference" or misuse of assessment results by 
outside agencies appear to be more apparent than 
real, at least thus far (a finding we*d like to see 
replicated in future surveys.) 

Despite the eariier mentioned relatively 
attenuated on faculty roles in CSAPs, strong 
faculty involvement in assessment was the most 
frequently cited helpful decision in establishing a 
comprehensive assessment program. 
Additionally, the variety of other successful 
resolutions and decisions given by institutions 
support the strong need for opportunities to 
share ideas and creative solutions to the complex 
issues raised by higher education assessment 
activities. 

Opportunities for sharing resources may 
also need to be pursued more rigorously by 
institutions in the future. (The need for more 
resources was the most frequently cited unmet 
need in our survey.) One way in which 
resources can be shared among institutions is 
through assessment networks, but thus far only 
10% of colleges and universities report 
belonging to such a consortium. Analysis and 
feedback of results, belter assessment 



instruments, greater faculty involvement, and 
better planning and goal setting are a few of the 
other unmet needs which institutions face as they 
strive to implement effective CSAPs, 

A Developmental Pattern of Assessment 
Programs? 

While this study includes a wide variety 
of CSAP origins, organizations, operations, and 
issues, present response trends and case-study 
reports from veteran assessment institutions 
seem to form enough of an empirical pattern to 
derive a typical nine-step CSAP development 
cycle: 

1. An institution's administration * with or 

without external assessment mandate 
pressure and/or meaningful faculty input - 
commits to the establishment of a CSAP as 
a primary indicator of the institution s 
effectiveness. 

2. A relatively latent "behind the scenes" period 

of information gathering, goal setting, 
planning, consensus-building, program 
design, resource seeking and trial-and- 
crror exploration of assessment models 
ensues, 

3. Then a concerted CSAP effort begins, 

supplementing existing data-gathering 
activities with "pilot" assessments in 
selected programs across the campus. 

4. Gradually the process effects of CSAP 

implementation are evidenced. Students, 
faculty, administrators, outside mandaters 
and others begin to be influenced by 
assessment activities (e,g., defining 
program objectives, finding and/or 
developing measures, piloting assessment 
strategies, analyzing preliminary evaluative 
feedback, preliminary reporting, etc.) 

5. Assessment refinement eventually matches 

program objectives with effective 
measures at a manageable cost sufficiently 
to implement an institution-wide CSAP, 

6. Finally product analysis of the CSAP begins 

to yield a definitive database on academic 
program quality, and the strengths and 
weaknesses in the curriculum, instruction, 
policies, and resources which are 
responsible for that level of quality. 

7. Reactions to those results come from 

program faculty, institutional 
administrators, assessment mandaters, and 
others. 
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8. Changes in educational programs and/or 

policies and/or CSAPs are made to 
enhance the institution's effectiveness. 

9. Those changes are, in turn, assessed,.. .and 

the higher education assessment cycle 
continues. 

According to the results of this study, the 
majority of U.S. colleges and universities (59%) 
are at Step 1 or 2, just feeling their way into the 
assessment movement. Another 30% are some- 
where between Steps 3 and 5, making a serious 
effort towards implementing a CSAP, but still 
wrestling with significant methodological, logis- 
tical, motivational and fmancial issues. A much 
smaller percentage - mostly those pioneering 
schools to whom those that follow owe such a 
debt of gratitude - are at Step 6 and beyond. 
And somewhere around 10% of American 
institutions either haven't gotten the word, are 
still trying to find a way around assessment, or 
are still at a pre-planning stage of CSAP 
implementation. 

Some Unanswered Questions on the 
Future of the Higher Education 
Assessment Movement 

As we've stressed throughout this 
report, it is premature to be making confident 
long-term predictions. But in the short-term - 
i.e., the next few years - the expansion of 
current CSAPs and assessment mandates, and 
continued new institutional commitments to 
CSAPs, seem assured. Still, we must be 
cautious, since even for most institutions with 
state and/or regional accreditation mandates the 
first rounds of assessment reports, self-studies, 
and team visits have just recently gotten 
underway. Until the assessment">report" 
>feedback loop has been closed, nothing is 
certain. 

Many important process questions 
remain to be answered if assessment growth is 
to be sustained. For example, will increased 
roles for faculty and students be achieved? Will 
administrator resolve remain firm? Can the 
resources needed for quality assessment be 
found? Are assessment process benefits 
sufficient to sustain motivation? Are the 
necessary CSAP quality control safeguards in 
place to assure useful assessment results? 

Soon more pressing assessment product 
questions will take center stage, and many of the 
answers will come from outside the institutions. 
How will state and accrediting mandaters react 



to positive CSAP results?., .to negative CSAP 
results? What will states or accreditors do if 
certain schools or programs refuse to implement 
CSAPs?. ..or do so in a perfunctory or clearly 
invalid manner? Will good programs and 
proficient facalty be rewarded? Will struggling 
but needed programs and overworked but 
underachieving faculty receive increased 
resources and support? Will unnecessary, 
inefficient, and unproductive programs suffer 
any meaningful sanctions? ..and what of poor 
faculty and unsuccessful administrators? What 
will happen to students who - because of CSAPs 
- show they haven't been learning what we've 
always: thought they were learning? Will CSAPs 
produce higher achievement but at the expense 
of student retention?. ..or student body 
diversity?. ..or faculty research 
productivity?. ..or academic creativity?. ..or 
what?! CSAPs' future will, at the very least, be 
interesting. 

Even broader issues and questions, 
which are beyond the scope of this survey, are 
perhaps equally likely to determine the longer 
term future of assessment in higher education as 
we move toward the 21st century. How 
substantial an effect, for example, will the 
current general economic downturn have on 
resources, allocated to higher education? Will 
education in general and higher education in 
particular continue to be highly valued by state 
decision-makers and private funding sources? 
Will the trend toward an increased emphasis on 
the quality of education and teaching versus 
research continue to grow and be supported by 
higher education institutions? In other words, 
can the educational and economic climate 
necessary to sustain this movement toward 
increased effectiveness be expected to last long 
enough to achieve its goals? 

Not one of these questions can be 
answered definitively at this time, nor should we 
expect to have sufficiently valid outcomes results 
from most institutions' CSAPs to draw informed 
conclusions for many years. Thus, as of 1990, 
the jury on the higher education assessment 
movement's long term future is still very much 
out. 

Some Contingent Predictions 

In our view, the next two to five years 
will be crucial to the assessment movement, and 
thereby to a large extent, to the future course of 
higher education in this country. Assessment 
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clearly has the potential to be a major factor in 
upgrading quality, but there are many important 
conditions yet to be met. IF CSAPs are 
implemented widely, efficiently, and 
validly. ..and IF the results are utilized wisely for 
program improvement... and IF state agencies, 
accreditors, the media and the public deal fairly 
with assessment processes and prod- 
ucts...THEN we are convinced that CSAPs will 
become such valued fixtures on college 
campuses that educators and students alike will 
wonder how we ever got along without them as 
long as we did. 

On the other hand, the higher education 
assessment movement certainly has potential for 
abuse. IF administrators don't involve faculty 
and students in the forefront of CSAPs, little 
meaningful progress should be expected. IF 
assessors aren't highly selective and careful with 
their CSAP methods and procedures, accurate 
results will be few and far between. IF institu- 
tions and funding sources aren't supportive of 
quality CSAP requirements, a costly and 
demoralizing triumph of form over substance 
may well ensue. IF assessment mandaters don't 
demand valid CSAPs, and substantially reward 
good programs £ind sanction bad ones. ..or IF 
administrators, the government, and/or the 



media turn our movement into a corruption of 
assessment (such has been the case with much 
of primary and secondary education's group 
test-based "assessment" programs)... THEN 
assessment runs the risk of becoming an 
academic and political debacle, with an anti- 
CSAP "baby-with-the-bathwater" counter- 
assessment movement not far behind. 

Given the high degree of assessment 
planning and implementation occurring in our 
nation's colleges and universities, future surveys 
on assersment in higher education will 1 xely 
find new perspectives, methods, processes, 
issues, and concerns. Given the high quality of 
educators we have working in assessment, new 
solutions to even the toughest questions will 
likely be forthcoming as well. This is entirely 
fitting, since assessment is, after all, a dynamic 
process of systematic self-examination intended 
to stimulate improvement. 

We are dedicated to doing our part in 
support of conscientious higher educators and 
CSAP assessors everywhere to see that 
assessment fulfills its promise, and hope this 
report and the other results of our study to 
follow are significant contributions toward that 
end. 
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Table 4. Percentage of CSAP Institutions That Assess Particular Objectives, by Method of Assessment 
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Dean of students 


9 


4. 


Director of institutional research 


7 


5. 


Dean of academic affairs 


7 


6. 


Chief executive officer 


5 


7. 


Vice president for student affairs 


4 


8. 


Other 


14 


9. 


No respt)nse 


3 



4 c 



1 4 
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Table 6. Day-to-day Operational Responsibility for 

Comprehensive Student Assessment Programs 



Perceniagc of 



Officer with Insiituiions 
Operational Rcsponsibiliiy (N=79()) 



1 . An adminisiraior t29f 

2. A faculty metnber 20 

3. A faculty ci>mniiitcc 10 

4. Other 6 

5. No response 3 



Table 7, Percentage of Institutions Reporting Faculty Participation 
in the Assessment Program, by Degree of Involvement 



Perccniage ol 



TYPE in PARTiC IPATION Insmuiions 
Degree of Faculty Involvemeni (N=79()^ 



1. PLANNING AND DHSIGNINCi 
THE ASSESSMENT PROGRAM 

a. Heavily involved A^i 

b. MiKlerately involved 25 
Somewhat involved 25 

d. Not at all involved 4 

2. IMPLEMENTING/OPERATING 
THE PROGRAM 

a. Heavily involved 34^/f 

b. Mixierately involved 13 

e. Somewhat involved 40 
d. Not at all involved 12 

3. EVALL ATING THE PROGRAM 

a. Heavily involved 2\Vr 

b. M(Klerately involved 34 

c. Somewhat involved 36 

d. Not at all involved 9 

4. DIRECTING THE PROGRAM 

a. Heavily involved 

b. Moderately involved 23 

c. Somewhat involved 45 

d. Not at all involved 13 



Niilc: Deiail iiu!> iioi sum lo 100 because ol rounding. 
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Table 8, Percentage Distribution of CSAP Leaders by Academic Discipline 



Primary 



Secondary 





OiscipliiK" 


L-eader 


U-adcr 


1. 


F-'iluLalion 








S(Kjal ;ind tx'havjdral scieiiLfs 


24 


IS 


.V 


Arts ami huiiianiiics 


21 


li 


4. 


Ph> sical sl icmkl's. iiialhciiiaiics. ciijiiiK'LTinj: 


7 


12 


S. 


F'rotcssional f ields 




4 


fi, 


Bi()l()j;ical SL'ifiKvs 


1 


1 


7. 


Other disciplines 


4 


6 


S. 


No response 


12 


31 



Table Percentage Distribution of Institutions, by Degree to Which Selected Challenges Pose f^roblems tor CSAPs 



A \ a 1 1 a b i 1 1 Si luicn t t' a c u 1 1 \ 

\\ ot wilul Moiiva- CSAF* Motiva 

AsscssDKMit iion/parli Coortli Finaiuial tu>n/parti- 

histrumcrus tipaiiiMi iiatton Supporl cipiilion 



( SAP ke 
suits hn 
Ptini, Kn 
hanccincnt 



Valid 
Analysis 
o\ Rcsuhs 



Int1 uenco 

Ironi 
Outside 
Agencies 



Misuse ol 
Results 
by Outside 
Agencies 



Ad mm 

slrative 

Support 



Misuse ol 
ResuIlN by 

Media, 
(iovt.ele. 



Any dc^iree (siiizht to 

vvry severe) ^ 7^^ 7^; 77 7.S 70 6^ 67 40 25 25 20 
Severe or very severe 

problem ^ 19 14 \^ 20 II II 8 9 4 3 2 

No problem I9 14 20 22 27 21 29 50 55 6S 5« 

No response 2 7 3 3 3 10 4 10 20 7 22 



Tahle H). Pcrcenlage oi Inslitutions That Idenlified Selected Problem Areas 
as "Most Severe" 



ln>lilv)tiiMis 

Problem Arc;i (N-7M()) 



1 


Siuilcn! inoti\ Jtion/participjtion 


l«' 


^ 


( iiuincial support 


\f^ 


V 


l atiilty iiiotivaiioii/partic ipatKM) 


IS 


4. 


Availability ol asscssnioni tnstruniciits 


l^> 


.S. 


tlsc (>r results lorprojirani cnhaiKcnicnt 


\} 


6. 


AJmiiiistrativo supix)rt 


1 


7. 


( (N)riliiiati(>ii ot the as .cssnicnt program 


(> 


H. 


Analysis atut interpretation ol results 


4 




liilluenee tr()m outsitle agencies 


-» 


to 


Misuse ot results by media, ^overnnteiu 


2 


II. 


Misuse ol results bv outside ajieneies. sueh as aeerediiini: 
boiltes. ete. 


1 


i:. 


( )ther 


6 



Nok*: Kc^pt>iulL-nis ucif .iskL^I lo kIlitIiI) 1 ol \\w I I LlLtlk'iij,'es *is llu* "n»nsi si'mtc" Simik' 
ulciililk'il I wo. oiliL'ts. onl> t>no. tuci one Ihiul Jkl Mi>l ulentily *u)> ol iIk' ctuik'n^cs as inosi 
sev fre ' 



Table 1 1 . Percentage of Institutions Citing Beneficial Effects of a 
Comprehensive Student Assessment Program (CSAP) 



Percentage of 
Institutions 



Beneficial Effect (N=79()) 



1 . Academic planning enhanced 3()7r 

2 . Students' efforts, feedback enhanced 29 

3. Standards upgraded; niorale enhanced K 

4. Administration's awareness, involvement, cooperation enhanced 4 

5 . Faculty interaction, cooperation enhanced 3 

6. Other I 

7. No response; too soon to respond 26 



Table 1 2. Percentage of Institutions Citing Detrimental Effects of a 
Comprehensive Student Assessment Program (CSAP) 



Percentage of 
Institutions 

Detrimental Effect (N=79()) 



1. 


Extra work for faculty and administration 


9<5f 


2. 


Costs increased; resources reallocated 


K 


3. 


"Turf" problems 


5 


4. 


Drop in morale 


4 


5. 


Other 


12 


6. 


No detrimental effects 


9 


7. 


No. response; too soon to respond 


53 
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AMERICAN 
gpUNQLON 
EDUCATIOH 



American Council on Education 
Winthrop College 

Higher Education Panel Survey No. 79 

Survey of Student Assessment Programs 




This questionnaire is designed to gather data regarding current efforts toward student assessment on campuses 
across the nation. At most institutions, comprehensive student assessment programs involve undergraduate 
students in the arts and sciences. Please complete this questionnaire for such a program unless your 
institution's program is directed to some other group of students, If the latter is the case, indicate here which 
students are involved in your assessment program. 



The term, student assessment, as used in this survey refers to measuring student progress toward 
the institution's educational goals, (e.g., competencies derived from general education programs and 
major areas of study, basic skills remediation, career preparation) and related student-development 
goals, (e.g., personal and social values and behavior). 

SECTION I. PROGRAM STATUS 

1, Is your campus under a state mandate to implement an assessment program? 

No. Yes. If yes. in which year did/does the mandate go into effect? 19 

2. Does your campus have a currently opt^rating comprehensive program to assess student learning and 
development? 



No. If no, go to item 3. 

3. What IS the status of planning for a comprehensive 
student assessment program on your campus? 

there are plans to implement an assessment 

program in or prior to Academic Year (AY) 
1 990-91. 

there are plans to implement a program m AY 
1991-92 or later. 

a student assessment program is currently 
under discussion but a target date for 
implementation has not been set. 

the campus does not plan to implement a 

student assessment program. Please explain 



Respondents who answer this item (No. 3) need not 
answer any of the following questions. Please go 
directly to the end of the questionnaire, complete the 
respondent information block, and return the form. 

Thank you for your assistance. 



Yes. If yes, please go directly to item 4. 

4 In which academic year was your student assess- 
ment program begun? 

Academic year 19_ - 

What were tne principal reasons for establishing your 
student assessment program? Check all that apply in 
boxes to the left of the reason, and rank the three 
mo:,t important reasons by putting numbers 1, 2. and 
3 in the blanks at the right for the first, second, and 
third most important reasons, respectively. 




Accreditation standards 
State law 

Governor's initiative 

Higher education commission's 
regulations 

Campus administration's decision 

Outside consultants* recommendations 

Faculty initiative 

Following other institutions' examples 
Following 'national trends" 
Other; specify. 
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5. For each of the program components listed (A through F), indicate which assessment methods are currently 
used. For each component, please circle the number($) for all the methods in use. If a component is not included 
in your campus* assessment program, circle the '0** in the ''None* column. 

Astettment Methods 

1. Commercially developed tests, e.g., ACT's CAAP« ETS* Academic Profile, etc. 

2. Locally developed tests/exams. 

3. Performance-based assessments, e.g., observations, simulations, demonstrations, practica. 

4. Student self-report methods, e.g., interviews, ratings, surveys. 

5. Available archival records, e.g., transcript evaluation, attainment rates, other records. 

6. Other methods; please describe at the bottom of the page* 



Assessment Method /Circle all that applv^ 

Com'l. Local Perfor- Self- Re- 
Assessment Program Components None Tests Tests mance report cords Other* 



A. Basic college-readiness skills (Measurmg 
student progress in reading, writmg, 
mathematics, etc.) 0 

B General education/liberal studies (Measuring 
student progress in the humanities, natural 
sciences, social sciences, international 
studies, etc.) 0 

C. Major field of study (Measuring student 

progress m degree or program specialization, 
e.g. biology, early childhood education, music, 
pre-law, etc.) 0 

Fjr each method useu, indicate the per- 
centage of vour departments that 
use that method. 



6 * 



6 • 



D. Career preparation/alumni follow-up 

(Measuring post*college adjustment, occu- 
pational success, graduate training, etc.) 0 

E Student personal growth & development 
(Measuring values, attitudes, social develop- 
ment, and/or other nonacademic changes 
attributable to the college experience.) 0 

F Other student outcomes, specify. 



2 
2 
2 



3 
3 
3 



4 
4 
4 



5 
5 
5 



6 ' 



6 • 

6 ' 
G • 
6 • 



'Other assessment methods, specify. 
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SECTION II. PROGRAM PERSONNEL AND ORGANIZATION 

6. Who has the primary executive responsibility for the student assessment program? Check only one. 
a. President/chief executive officer 

b. Vice president for 

c. Chief academic officer 

d. Dean of 

e. Director of institutional research 

f. Other; please specify. 

7. Who is in charge of the day-to-day operation of the assessment program? Check only one, 

a. An administrator; title: 

b. Faculty member($); title: 

c. Faculty committee; name: 

d. Other; describe and give title/name 

8. For the one or two most directly involved leaders of your campus* assessment program • 



a. indicate the academic discipline and highest earned 
degree. 

Leaoer #1: 

Discipline 

Highest earned degree 

Leader #2: 

Discipline 

Highest earned degree 

b. From where were they chosen? 

Leader Leader 
#1 #2 

7. From within your institution 

2. From another higher 

education Inslitulion 

3, From a setting outside higher 

education. Please describe 

the settings for: 

Leader #1 



c. For each of the leaders, indicate their prior 
Iraining and experience relevant to higher education 
student assessment. Check all that apply. 

Leader Leader 
#1 #2 

r Took a degree program with an 
emphasis in assessment or 
measurement 

2. Had a degree in education 

or higher education 

3. Had attended workshops and/or 

seminars or conferences 

on assessment 

4. Had experience in assessment 

as applied to academic 

discipline(s) 

5. Had cxporience in assessment 

at the institutional level 

6. Had other relevant experience; 

specify. 



Leader #2 



7. Had no prior experience 
with assessment programs 
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9, Do you have an **Office of Student Assessment" or similar coordinating center for student assessment 
efforts on your campus? 



No, 

If no, go to question 10, 



Yes. If yes, what is the exact name of that office? 



c. How is the office staffed? Indicate the number of each type of staff in full-time-equivalents (PTEs), 
Categorize individuals by their institutional claetlflcetion. not by the type of work they are doing for the 
assessment program. For example, if a graduate student is performing clerical duties, count him/her as a 
graduate student; or if an English instructor is helping to administer the program, classify him/her as a ^acuity 
member. 



(1) Administrators 

(2) Faculty 

(3) Secretarial/clerical 



JTE 
FTE 



(4) Graduate/student assistant 

(5) Other; specify 



FTE 
_FTE 
FTE 



d. What assessment functions does the office perform? Check all that apply. 



(1) Planning 

(2) Coordination 

(3) Budgeting 

(4) "Hands on" assessment 

implementation 

(5) Technical assistance 

(6) Research and development 

(7) Program evaluation 

(8) Consultation with administration 



(9) Consultation with faculty 

(10) Liaison with students 

(11) Line authority over - 



(12) Other; specify. 



10, What other organizational structure(s) are di fgp^Jy involved in the planning, evaluation, or advisory 
activities of your student assessment program? Check all that apply and indicate the specific name of the 
office, committee, or group involved, 

a. Administrator groups; name 



b. Faculty committee/council; name^ 

c. Student committee/council; name 

d. Consultants; specify fields 

e. Other; specify 



11. Is your institution part of an assessment consortium? 

No. Yes. If yes, name it and indicate the number and type of members. 



12. Circle the number that corresponds most closely to the degree to which faculty were/are 

involved in - 

Heavily Moderately Somewhat Slightly 
Involved 



ERLC 



Involved Involved 



Not at all 
Involved Involved 



a. Planning & d<)signing the assessment program 5 

b. Directing the assessment program 5 

c. Implementing/operating the assessment program 5 

d. Evaluating the assessment program 5 
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4 
4 
4 
4 



3 
3 
3 
3 



2 
2 
2 
2 
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SECTION III. EVALUATION OF THE ASSESSMENT PROGRAM 



13. Below are a number of the challenges faced in the implementation of a successful student assessment 
program. Indicate the degree to which each has been a problem in the development of your program by circling 
the appropriate number. 



Challenge 



Very 

No Slight Moderate Severe Severe Not 
Problem Problem Problem Problem Problem Applic, 



a. 
b. 
c. 
d. 
e. 



rinanciai suppon i 


2 


3 


4 


5 


6 


Administrative support 1 


A 

2 


A 

3 


4 


5 


A 

6 


racuiiy moiivauon/pariicipaiion i 


o 

£ 


o 


4 


5 


A 

6 


oiUQeni moiivaiion/pariicipaiion i 


2 


A 

3 


4 


5 


A 

6 


miiuence irom uuisiue agencies, 












^eguiaiions, manoaies, eic. i 


2 


A 

3 


4 


5 


6 


^oorainaiion oi aii aspecis oi ine 












as^woot 1 lo 1 1 1 k^f wUf ai 1 1 1 


£, 


o 


*♦ 


e 
9 


c 
o 


Availability of valid assessment instruments 












and methods 1 


2 


3 


4 


5 


6 


Mridiybis ana inierpreiaiion oi assessment 












results 1 


2 


3 


4 


5 


6 


Use of assessment results for 












program enhancement 1 


2 


3 


4 


5 


6 


Misuse/misinterpretation of results by outside 












agencies, accrediting or regulatory bodies 1 


2 


3 


4 


5 


6 


Misuse/misinterpretation of results by media. 












governmental bodies, public, etc. 1 


2 


3 


4 


5 


6 


Other challenges; specify. 














2 


3 


4 


5 


6 




2 


3 


4 


5 


6 




2 


3 


4 


5 


6 



14, For the most severe challenges noted in question 13, list the approaches your institution used in meeting 
them, and indicate whether each approach was particularly successful or unsuccessful. 

The approach was 
success not sue* 

Challenge: ful cessful 

Approaches used to meet the challenge. 



Challenge; 

Approaches used to meet the challenge. 



15. Wr.at effects has your assessment program had on the effectiveness of your institution's educational 
program thus far? 

Main beneficial effects Main detrimental effects 



ERLC 



23 



16. What are your institution's three greatest unmat needs in implementing a successful student assessment 
program and using the results to enhance your educational program? 

a. Greatest need. 



b. Second greatest need. 



c. Third greatest need. 



17. What do you consider to be the two most helpful decisions or actions that were made in setting up your 
assessment program? 



18. How would you rale the quality of the student assessment program on your campus? 

a. Excellent cl. Fair 

b. Very good Q Poor 

c. Good Very poor 

_„ g- No basis to judge 

Please use the remaining space for any comr entc^ clarifications, or explanations that would help us get an 
accurate picture of your student assessment program. 



Thank you for your assistance. 

Please return the questionnaire in the attached 
postage*paid preaddressed envelope, and mail it 
by December 22, 198g to: 

Higher Education Panel 
American Council on Education 
One Dupont Circle 
Washington, DC 20036 



ERIC 



Please keep a copy of this form for your records. 
Name of 

Respondent 

Title 

Telephone ( ) 

3 A 
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APPENDIX B 
TECHNICAL NOTE 



The Higher Education Panel (HEP) forms the basis 
of an ongoing survey research program created in 1971 by 
the American Council on Education. Its purpose is to 
conduct Kpeciali/.ed surveys on topics of current policy 
interest to the higher education community. 

The Panel is a disproportionate stratified sample 
numbering 1.040 colleges and universiti , divided into 
two half-panels of 520 institutions each. The sample 
was drawn from the more than 3,200 colleges and 
universities lisied in the Education Direcr<>r\\ C<>llei^es 
and Universities published by the U. S. Department of 
Education's National Ceiiter for Education Statistics 
(NCES). The Panels stratification dr?sign is based 
primarily upon institution type, control, and si/e. For 
any given survey, either the entire Panel, half-panel or 
an appropriate sub-group is used. 



The survey mechanism relies on a network of 
campus representatives at institutions whose presidents 
have agreed to participate. The representatives receive the 
Panel questionnaires and direct them to the most 
appropriate campus official for response. 

The sample for this study consisted of 455 
institutions. Removed from one of the half-panels were 
graduate-only institutions along with all of the 
"speciali/ed" institutions, i.e., separately organized 
colleges of medicine, religion, law. business, health 
sciences, fine arts, etc. 

Oaestionnaires were mailed to each member of the 
half-panel in early November 1989. A follow-up mailing 
was sent in Janu-^ry 1990, and extensive follow-up 
telephone calls wert made in the period March-May 1990, 



Table B-l. Stratification Design 



Cell 


Type oi Insiiiutum 


F.nro!lmeni 


Population 


Respon- 
dents 




AM institutions 




2.fil9 


357 


1 


Lar^c public doctoratc-^rantin^ 


a 




39 


2 


Large private doctoralc-grainini: 


a 


5« 


23 




Large public comprehensive 


a 


92 


34 


4 


Large private comprehensive 


a 


26 


iO 


7 


Large public 2-ycar 


a 


43 


13 


K 


Public comprehensive 




56 


16 




f*ublic comprehensive 


< 5M)i) 


\i)H 


19 


10 


Private comprehensive 


<9,0(H) 


126 


20 


1 1 


Public baccalaureate 


<y. ()()() 


127 


17 


12 


Private baccalaureate 


l.35()-H.999 


166 


20 


13 


Private baccalaureate 


< 1,350 


446 


2K 


17 


Public 2-yr. academie/comp'hensive 


6. ()()()-«. 949 


5 5 


13 


IK 


Public 2-yr. acadcmic/comp hcnsive 


4.()()()-5,999 


72 


13 


ly 


Public 2-vT. academic/comp'hensive 


2,()()()-3,999 


155 


22 


20 


Public 2-yr, academic/comp'hensive 


<2A)iH) 


333 


21 


21 


Private 2-yr. acad,/comp*hensive 


<9,(H)() 


129 


10 




Public 2 year occupational 


2,50()-S,y99 


63 


14 




Public 2'year occupational 


<2,5()() 


221 


19 


24 


Private 2-year occupational 


<9,()()0 


240 


6 



a Insiiiuiions that meet one or more of the three following criteria: (I ) total full-time equivalent (PTE) I9KI enrollment greater 
than 8,999: (2) graduate PTE I9HI enrollment greater than 749: (3) FY 1979 educational and general expenditures of i35 million 
or more. 
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These efforts netted usable questionnaires from 357 half- 
panel members by the end of May 1990. resulting in an 
overall response rate of 78 percent. 

Data from the responding institutions were weighted 
and adjusted for item and institutional nonresponse within 
each of the cells in the stratification design shown in 
Table B-l. These procedures provided estimates 
representative of the national population of 2,619 



universities, 4-year colleges, and 2-year colleges (the 
3,200 institutional universe minus graduate only and 
^specialized' institutions). 

Table B-2 compares survey respondents and 
nonrespondenis on several variables. Response rates were 
greater than average at comprehensive institutions. They 
were below average at independent baccalaureate and two- 
year colleges. 



Table B-2, Comparison of Respondents and Nonrespondents to the Survey Questionnaire, by Control and Type of 
Institution 



Con!n>l and 

Type (>r Institution 



Respondents 



Non- 
respondents 



Response 
Rate 



Total 

Control 
Public 
Independent 

Control and Type of Institution 
Public ujiiversitv 
Independent university 
Public comprehensive um\, or colleize 
Independent compreliensi\ e uni\, or coll. 
Public baccalaureate Collei!c 
Independent ci^lle^ie 
Public lur year colle^ie 
Independent two-year ct>!lei:e 



100, 0 100.0 7K.5 

67 2 61.2 90.0 

32.8 3K,S 75.5 

10.9 11.2 78/) 
6.4 6.1 79.3 

19.3 16.3 HI. 2 

8.4 7.1 8M 
4.8 5.1 77,3 

13.4 16.3 75,0 
32.2 28.6 80.4 

4.5 9,2 64.0 
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